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We study the flavor-changing-neutral-current process c ujj,^ n~ using 1.3 fb~^ of pp collisions 
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at ^/s — 1.96 TeV recorded by the DO detector operating at the Fermilab Tevatron CoUider. We 
see clear indications ol the D'^ and — >■ (j)Tv^ — > /x^/i~7r+ final states with significance greater 
than four standard deviations above background for the D'^ state. We search for the continuum 
decay of — > Tr^/i^/i" in the dimuon invariant mass spectrum away from the (p resonance. We 
see no evidence of signal above background and set a limit of B{D^ —+ fi'^ fj,~) < 3.9 x 10~^ at 
the 90% C.L. This limit places the most stringent constraint on new phenomena in the c ufi^ 
transition. 

PACS numbers; 13.20.Fc, ll.30.Fs, ll.30.Hv, 12.15.Mm 



Many extensions of the standard model (SM) provide 
a mechanism for flavor-changing-neutral-current (FCNC) 
decays of beauty, charmed, and strange hadrons that 
could significantly alter the decay rate with respect to 
SM expectations. Since FCNC processes are forbidden 
at tree level in the SM, new physics effects could be- 
come visible in FCNC processes if the new amplitudes 
are larger than the higher-order penguin and box dia- 
grams that mediate FCNC decays in the SM. In B meson 
decays, the experimental sensitivity has reached the SM 
expected rates for many FCNC processes. In contrast, 
GIM suppression [l| in D meson decays is significantly 
stronger and the SM branching fractions are expected to 
be as low as 10" This leaves a large window of 

opportunity still available to search for new physics in 
charm decays. There are several models of new phenom- 
ena such as SUSY i?-parity violation in a single coupling 
scheme ^ that lead to a tree level interaction mediated 
by new particles, or little Higgs models with a new up- 
like vector quark \3\ that lead to direct Z cu couplings. 
In both scenarios deviations from the SM might only be 
seen in the up-type quark sector, motivating the exten- 
sion of experimental studies of FCNC processes to the 
charm sector. 

In this Letter we report on a study of FCNC charm 
decays including the first observation of the decay Df 

/i+/i^7r+ and the first evidence for the decay 
/)7r+ —^ ^+/.t~7r"*" by requiring a dimuon mass win- 
dow around the nominal (/> mass. The inclusion of charge 
conjugate modes is implied throughout the text. At the 
reported level of statistics, we expect no contributions 
from two body D^^^ decays due to the smaller rj, 
p, and uj branching fractions and^the smaller 77, p, and 
Lu — + /i^/i^ branching fractions 6]. The search for the 
c — > up^p^ transition in the decay — > ■n'^ p'^ p~ is 
performed in the continuum region of the dimuon invari- 
ant mass spectrum below and above the resonance. We 
focus on the continuum decay as opposed to similar 
£)+ or Ac decays due to the longer lifetime and higher 
production fraction of the D+ meson. The study uses 
a data sample of pp collisions at = 1.96 TeV corre- 
sponding to an integrated luminosity of approximately 
1.3 fb~^ recorded by the DO detector operating at the 
Fermilab Tevatron Collider. Similar studies have recently 
been published by the FOCUS 0] and CLEO-c [1] col- 
laborations, and preliminary results have been presented 



by the BaBar collaboration. 

DO is a general purpose detector described in detail in 
Tol . 11 1 . Charged particles are reconstructed using a 
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Ref. 

silicon vertex tracker and a scintillating fiber tracker lo- 
cated inside a superconducting solenoidal coil that pro- 
vides a magnetic field of approximately 2 T. Photons 
and electrons are reconstructed using the inner region of 
a liquid argon calorimeter optimized for electromagnetic 
shower detection. Jet reconstruction and electron iden- 
tification are further augmented with the outer region of 
the calorimeter optimized for hadronic shower detection. 
Muons are reconstructed using a spectrometer consisting 
of magnetized iron toroids and three super-layers of pro- 
portional tubes and plastic trigger scintillators located 
outside the calorimeter. 

The analysis is based on data collected with dimuon 
triggers. The DO trigger is based on a three-tier sys- 
tem. The level 1 and 2 dimuon triggers rely on hits in 
the muon spectrometer and fast reconstruction of muon 
tracks. The level 3 trigger performs fast reconstruction of 
the entire event allowing for further muon identification 
algorithms, matching of muon candidates to tracks recon- 
structed in the central tracking system, and requirements 
on the z position of the pp interaction. 

The selection requirements are determined using 
PYTHIA [I4I Monte Carlo (MC) events to model both cc 



and hh production and fragmentation. The evtgen [13| 
MC is used to decay prompt D mesons and secondary D 
mesons from B meson decay into the (/)7r+ and p'^p^n^ 
intermediate and final states. The detector response is 
modeled using a GEANT [i3| based MC. The dimuon trig- 
ger is modeled using a detailed simulation program incor- 
porating all aspects of the trigger logic. Backgrounds are 
modeled using data in the mass sideband regions around 
the D meson mass of 1.4 < TO(7r+/i+/i~) < 1.6 GeV/c^ 
and 2.2 < m{Tr+p+p-) < 2.4 GeV/c^. 

Muon candidates are required to have segments recon- 
structed in at least two out of the three muon system 
super-layers and to be associated with a track recon- 
structed with hits in both the silicon and fiber trackers. 
We require that the muon transverse momentum px be 
greater than 2 GeV/c and the total momentum p to be 
above 3 GeV/c. The dimuon system is formed by com- 
bining two oppositely charged muon candidates that are 
associated with the same track jet form a well re- 
constructed vertex, and have an invariant mass ni{p'^ p~) 



5 



15000 



> 

I 10000 

V) 



a 

W 5000 



1 1 1 1 1 1 1 1 
/ 1 


1 1 1 1 1 1 1 1 

D0, 1.3 fb"* 


/col 






/A 








... 1 ... . 


... 1 ... . 



1.40 0.65 



0.90 



1.15 



m()a'"n") (GeV/c ) 



1.40 



FIG. 1: The inclusive m(/i^^~) invariant mass spectrum. 
The fitting function includes components from the rj, p, uj, 
and (h resonances. 



the angular separation between the pion and the dimuon 
system in rj-cj) space, (Ai?^)^ = (A?7)^ + (A0)^, which is 
typically small for true candidates. 
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below 2 GeV/c^. The dimuon mass distribution in the 
region of the light quark-antiquark resonances is shown 
in Fig. [T] Maxima corresponding to the production of 
UJ and (f) mesons are seen. The p is observed as a broad 
structure beneath the uj peak, and there is some indica- 
tion of rj production as well. For the initial search for 
resonance dimuon production we require the fJ-^ fJ-" mass 
be within ±0.04 GcV/c^ of the nominal (p mass and re- 
determine the muon momenta with a mass constraint 
imposed {ij which improves the /i~'"/i~7r+ invariant mass 
resolution by 33%. 

Candidate D'^^^ mesons are formed by combining the 
dimuon system with a track that is associated with the 
same track jet as the dimuon system, has hits in both 
the silicon and fiber trackers, and has pr > 0.18 GeV/c. 
The pion impact parameter significance 5,7, defined as 
the point of closest approach of the track helix to the 
interaction point in the transverse plane relative to its 
error, is required to be greater than 0.5. The invariant 
mass of the three body system must be in the range 1.4 
GeV/c^ < TO(7r+/i+/i-) < 2.4 GeV/c^. The three parti- 
cles must form a well-reconstructed D meson candidate 
vertex displaced from the primary vertex. The transverse 
flight length significance iS_d, defined as the transverse 
distance of the reconstructed D vertex from the primary 
vertex normalized to the error in the reconstructed flight 
length, is required to be greater than 5. The coUinearity 
angle O d , defined as the angle between the D momentum 
vector and the position vector pointing from the primary 
to the secondary vertex, is required to be less than 500 
mrad. In events with multiple pp collisions, the longitu- 
dinal track impact parameters are used to reject muons 
and tracks produced in the secondary pp interactions. In 
events with multiple D candidates, the best candidate is 
chosen based on the Xvtx the three track vertex and 



FIG. 2: The in{Tr'^ jj.^ fi") mass spectrum in the 0.98 < 
m{fi^fi~) < 1.06 GeV/c^ 4> mass window. The result of a 
binned likelihood fit to the distribution including contribu- 
tions for , Df , and combinatoric background is overlaid 
on the histogram. 

The resulting 7r+^+/i^ invariant mass distribution is 
shown in Fig. [2l The D^^^ 07r+ — > /.t+^~7r+ signal is 
extracted from a binned likelihood fit to the data assum- 
ing possible contributions from D"*" and initial states 
as signal and from combinatoric background. The 
component is modeled by a Gaussian function with the 
mean and standard deviation as free parameters. The 
component is modeled as a Gaussian function. The 
difference in means between the D'^ and Df Gaussian 
functions is constrained by the known mass difference 
and the ratio of the standard deviations is constrained 
to the ratio of masses Q. The background is mod- 
eled as an exponential function with floating parameters. 
The normalization of all functions are free parameters. 
The fit yields 254 ± 36 D+ candidates and 115 ± 31 D+ 
candidates. The statistical significance of the combined 
Df and signal is 8 standard deviations above back- 
ground. The significance of the yield, treating both 
the combinatorial and Df candidates as background, is 
4.1 standard deviations. 

The relative efficiency of the and channels is 
determined separately for prompt D mesons produced 
in direct pp cc + X processes and D mesons from B 
meson decay and combined using the measured prompt 
fractions [3 e+ = Z+e+^^pt + (1 - /p+)4^D' where 
f^prompt is the efficiency for prompt mesons, e^^jj is 
the eiiiciency for D+ mesons from B meson decay, and 
is the fraction of prompt mesons; we use equivalent 
expressions for Df mesons. The yield ratio is related to 
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TABLE I: External inputs to the yield ratio calculation. All 
numbers have been corrected for the most recent 
K^Tv^TT~ and (j)-K^ branching fractions '6|. 



/p+ [16] 

n [16] 



0.891 ± 0.004 
0.773 ± 0.038 



0.40 ±0.09 



the ratio of branching fractions by 



n{D+) f+_^^ f. 



fi fj. vr 



n{D+) f^^o /+ B{Dt ^ 4>t^+)B{^ ^ a^+A^") ' 



TABLE IL Final requirements for the background suppression 
variables along with the number of candidates surviving each 
requirement for the continuum — > Tr"*"^"*"/!" analysis. 



Preselection 

Pt{t^) 
Sd 

Id 
Qd 

X^tx (3 DOF) 
Signal window 



Requirement 



Surviving candidates 



< 2.6 

> 0.4 GeV/c 

> 9.4 

> 1.8 

> 0.7 

< 7 mrad 

< 2.6 

±2(7 



154203 
154131 
127027 
69817 
51736 
24742 

962 

212 
19 



where /^^l,^ is the fraction of Z?+ mesons produced in c 
quark fragmentation, and is the equivalent fraction 

for Df mesons [13] ■ The relevant numbers are hsted in 
Table HI The efficiency ratio is determined from MC to 
be eVe+ = 0.70 ±0.06 (stat ± sys). The difference from 
unity is caused by the lifetime difference between 
[cT — 147.0 ^m) and _D+ (cr = 311.8 ^m) mesons, and 
the systematic uncertainty is dominated by uncertainties 
in the resolution modeling of So and St^. 

Using the efficiency ratio, production fractions, and 
the D'^ — > (/)7r+ and —> branching fractions 

gives B{D+ </)7r+ ^+^-7r+) = (1.8 ± 0.5 (stat.) ± 
0.6 (sys.)) X 10"^, which is consistent with the expected 
value of (1.86 ± 0.26) x 10^^ given by the product of 
the D'^ (f>Tr~^ and (p branching fractions and 

other recent measurements d, Q . The systematic uncer- 
tainty is overwhelmingly dominated by the uncertainty 
in the 07r+ branching fraction that enters both 

the normalization and f^^jj- 

We now turn to the search for the continuum de- 
cay of D"*" 7r+/i+/i^ mediated by FCNC interactions. 
We study the dimuon invariant mass region below 1.8 
GeV/c2, excluding 0.96 < m{iJ+ iJ.') < 1.06 GeV/c^. 
Backgrounds are further reduced by requirements on the 
Sd, Stt, ©d, Xvtxj ^"^d ARjr variables defined above. The 
pion transverse momentum pt{t^) and the isolation de- 
fined as To = p{D) I ^Pcone wherc the sum is over tracks 
in a cone centered on the D meson of radius Ai? — 1 
are also used. The final requirements are chosen using a 
random grid search [l8| optimized using the Punzi (l9j | 
criteria to give the optimal 90% C.L. upper limit. The 
final requirements along with the number of candidates 
surviving each requirement are hsted in Table [ill 

The 7r+/x+/i^ invariant mass distribution in data for 
the dimuon invariant mass region below 1.8 GeV/c^ 
excluding 0.96 < m{fi~^fi~) < 1.06 GeV/c^ is shown 
in Fig. [3] The Z?+ signal region contains 19 events. 
The combinatorial background in the signal region is 
estimated by performing sideband extrapolations to be 
25.8 ± 4.6 events. The uncertainty reflects the range in 
the background estimation from variation in the back- 
ground shape across the 'K~^fi'^fi~ mass spectrum. The 



probability of the background fluctuating to the mea- 
sured event yield or fewer events is 14%. 




2.40 



m(n'"|.tV) (GeV/c ) 



FIG. 3; Final tv'^ fj,^ fi~ invariant mass spectrum. The ±2cr 
D'^ signal region, within the dashed lines, contains 19 events. 
The background level determined from the sidebands is 25.8 ± 
4.6 events. 



We normalize the results to the 



/i''"/z~7r+ signal instead of the larger 13+ signal to avoid 
the uncertainties associated with the _D+ and 13+ pro- 
duction fractions. We use the product of the known 



D+ 



and 



branching fractions [i- The 



Tr+zi+zi chan- 



signal efhciency ratio between the 
nel in the final sample and the 13+ /i+/i^7r+ 
channel in the preselection sample is determined from 
MC to be (5.4 ± 0.8)%. The inputs to the limit cal- 
culation are summarized in Table IIIII The systematic 
uncertainty is dominated by the modeling of the vertex 
resolution particularly in the Xvt-x requirement. Using 
this, we find 



S(Z3^ 



B{D+ -> <j)TT+) X 6(0 -> ^i+fi-) 



< 2.09, 90% C.L. 



The limit is determined using a Bayesian technique [20| 
Using the central value of 13+ (/)7r+ and (p fJ-'^IJ^' 



7 



TABLE III: Inputs to the B{D+ 7r+^i+/i") upper limit 
calculation and resulting upper limit at the 90% and 95% 
C.L. 



—> -k'^ fi'^ candidate yield 
Background expectation 

— > (/itt"*" — > -K^ fj,'^ candidate yield 
Relative efficiency 
B{D+ (t>-K+) 

Single event sensitivity 



19 events 
25.8 ± 4.6 events 
115 ± 31 events 
0.054 ± 0.008 
6.50 X 10"^ 
2.86 X 10"'* 
3.0 X 10"^ 





TT+M+M") 95% C.L. 


< 6.1 X 10"" 


B{D+ 


^ n+n+lj,-) 90% C.L. 


< 3.9 X 10"'' 



branching fractions gives 



< 3.9 X 10" 



90% C.L. 



This is approximately 30% below the limit one would ex- 
pect to set given an expected background of 25.8 ± 4.6 
events. The single event sensitivity, given by the branch- 
ing fraction one would derive based on one observed sig- 
nal candidate, is 3.0 x 10"^. 

In conclusion, we have performed a detailed study of 
and Df decays to the Tr+zx+zi" final state. We 
clearly observe the Df 07r+ intermediate state and 
see evidence for the D'^ 07r+ intermediate state. The 
branching fraction for the _D+ (f>TT^ — > 7r"'"/Lt final 
state is consistent with the product of D'^ — > (jjir^ and 
(j) — > fJ-'^ fJ-~ branching fractions. We have performed a 
search for the continuum decay of — )■ 7r+/i+/i^ by ex- 
cluding the region of the dimuon invariant mass spectrum 
around the (j>. We see no evidence of signal above back- 
ground and set a hmit of B{D^ — > 7r+^+/i~) < 3.9x 10~^ 
at the 90% C.L. This is the most stringent limit to date 
in a decay mediated by a c — > u^,^ jjT transition. Al- 
though this is approximately 500 times above the SM 
expected rate, it already reduces the allowed parameter 
space of the product of SUSY i?-parity violating cou- 
plings A22fc X 0. However, it is still an order of mag- 
nitude above the expected level from little Higgs mod- 
els 0. 
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